A surgical system for automatic registration, stiffness mapping and dynamic image overlay

In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.Comment: International Symposium on Medical Robotics (ISMR 2018

Probabilistic Approaches for Pose Estimation

<p>Virtually all robotics and computer vision applications require some form of pose estimation; such as registration, structure from motion, sensor calibration, etc. This problem is challenging because it is highly nonlinear and nonconvex. A fundamental contribution of this thesis is the development of fast and accurate pose estimation by formulating in a parameter space where the problem is truly linear and thus globally optimal solutions can be guaranteed. It should be stressed that the approaches developed in this thesis are indeed inherently linear, as opposed to using linearization or other approximations, which are known to be computationally expensive and highly sensitive to initial estimation error. This thesis will demonstrate that the choice of probability distribution significantly impacts performance of the estimator. The distribution must respect the underlying structure of the parameter space to ensure any optimization, based on such a distribution, produces a globally optimal estimate, despite the inherent nonconvexity of the parameter space. Furthermore, in applications such as registration and 3D reconstruction, the correspondence between the measurements and the geometric model is typically unknown. In this thesis we develop probabilistic methods to deal with cases of unknown correspondence. While the resultant formulation does not guarantee global optimality, it increases the basin of convergence. Another contribution of this thesis is the development of a generalized framework for probabilistic point registration. By setting functional parameters such as sensor noise and prior uncertainties appropriately, our framework captures many prior registration methods. Additionally our framework is also capable of predicting scope for improvement in the existing algorithms, which are then verified experimentally. To tie the ideas together, we present results in the context of surgical robotics - in particular we demonstrate a surgical system that is capable of performing real-time tumor localization, hand-eye calibration, registration of preoperative models to the anatomy, and augmented reality.</p